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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in 
the application. With this amendment and as reflected in this listing of claims, 
original claims 1-18 which now stand rejected, have been canceled. New claims 
19-23 are now presented: 

Listing of Claims: 

1. (Canceled). 



2. (Canceled) 

3. (Canceled). 

4. (Canceled). 

5. (Canceled). 

6. (Canceled). 

7. (Canceled). 

8. (Canceled). 

9. (Canceled). 

10. (Canceled). 
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1 H.(Canceled). 

1 12. (Canceled). 

1 13. (Canceled). 

1 14. (Canceled). 

1 15. (Canceled). 

1 16. (Canceled). 

1 17. (Canceled). 

1 18. (Canceled). 

1 19. (New) A method for recognizing compound events depicted in video 

2 sequences, said compound events being determined from occurrences of 

3 primitive events depicted in the video sequences, wherein the compound events 

4 are defined as a combination of the primitive events, the method comprising the 

5 steps of: 

6 (a) defining primitive event types, said primitive event types including: x = 

7 y; Supported(x); RigidlyAttached(x, y); Supports(x, y); Contacts(x, y); and 

8 Attached(x, y); 

9 (b) defining combinations of the primitive event types as a compound 

10 event type, said compound event type being one of: PickUp(x,y,z); 

11 PutDown(x,y,z); Stack(w,x,y,z); Unstack(w,x,y,z); Move(w,x,y,z); 

12 Assemble(w,x,y,z); and Disassemble(w,x,y,z); 
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13 (c) inputting, a series of video sequences, said video sequences depicting 

14 primitive event occurrences, such occurrences being specified as a set of 

15 temporal intervals over which a given primitive event type is true; and 

16 (d) determining, the compound event occurrences, such occurrences 

17 being specified as the set of temporal intervals over which the compound event 

18 type is true, wherein the sets of temporal intervals in steps (c) and (d) are 

19 specified as smaller sets of spanning intervals, each spanning interval 

20 representing a set of all sub-intervals over which the primitive event type holds 

21 and wherein the spanning intervals take the form a [ r [i 9 j] S9e [kJW , where 

22 a 9 fi 9 y 9 5 9 e 9 and £ are Boolean values, i 9 j 9 k 9 and / are real numbers, 

23 « [ r [*\yL> e [^J^]/? represents the set of all intervals a [p 9 q] fi where i< y p < 5 j 

24 and k< e q<£ /, a [p 9 q] p represents the set of all points r, where p< a r< p q , and 

25 x < e y means x < y when 0 is true and x < y when 9 is false. 



1 20. (New) The method according to claim 19, wherein the compound event type 

2 in step (b) is specified as an expression in temporal logic. 

1 21 . (New) The method according to claim 20, wherein the temporal logic 

2 expressions are constructed using the logical connectives V,3,v, A^O^and -i , 

3 where R ranges over sets of relations between one-dimensional intervals. 



1 22. (New) The method according to claim 3, wherein the relations are =, <, >, m, 

2 mi, o, oi, s, si, f, fi, d, and di. 



1 23. (New) The method according to claim 22, wherein the compound event 

2 occurrences are computed through the use of the following set of equations: 

3 s(M 9 p(c l9 ... 9 c n )) A {i| p(c X9 ... 9 c n )@\eM} 

4 ^OvT) A e(M 9 Q>)\je(M 9 *¥) 

5 e(MyxO) A U - U iifV--ni n 

— A/,0[x:=c, ]) i n ee (M ,<D[ x:=c„ ]) 
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6 
7 

8 

9 
10 

11 

12 
13 



15 

16 
17 

18 
19 

20 



21 



where C(M) = {c,,...,c„} 
e(M,3x&) A U s(M,Q>[x:=c\) 

— ceC(M) 

e{M,^<s>) a u - u i;n--ni; 



where, 



14 aiwwm^ 



where f(M,0) = {i,,...,i n } 
efM.OA^ A (J U LU(i,r,j) 

fCM.Oj.O) A U U3)(r,i) 



where / = min (y , /) 
k' = max(A: ? /) 
y' ~ y ai ^ -co 

S' = 8 a min(y,/) ^ oo a (7 < / v ^ a a a >3) 
e' =g Amax(&,/) ^ -oo a (A: > ivy a/3 Aa) 
= £ a/*oo 
when z < / a A:' < / a 

[,=/ -> (y' A £<)] A [*' = / -> (€' A^')] A 

[i = /-»(a A /?)]A 

i ^ OO A y f ^ -OO A ^ 00 A / ^ -00 

{} otherwise 



«, [ ri ['1 > J\ U > e , ft > A ^ ] A H a2 [ rj [i 2 , j 2 ] Sj » 6j ft , / 2 ^ ] A A 

< ai [ r [ max 0'i , i 2 ), min(y, , y 2 )k , 6 [max(*, , fc 2 ), minft , l 2 )] c \ > 

where y = \ \ 





/, > i 2 




h = h 




h <h 




h <h 


< A ^ 2 


h =J 2 




j\ > h 
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22 



23 



24 

25 

26 
27 
28 
29 



30 



31 
32 



33 



34 
35 
36 
37 
38 
39 

40 













^1 ^ ^2 




/,</ 2 




/, = / 2 




/,>/ 2 



when a, = a 2 a /?, = fi 2 
{} otherwise 



,[,[*'» ]j£ 



r<«[ T [-«>.»)rn-[-^>.*]-]>i>Ul 

<« [ T [-°°. »L r >T t" 00 ' °°]t ]fi > U 

>u 
>u 

CM- 00 ' 00 ]™ [- 00 . 00 ] T ]^>U 
[ T [-°°> °°]t »T t -00 ' °°] T > 

S P 3n (a, [„ ft > 7, L, > If , ] A >* 2 [ ri [*2 > 72 l 2 > Z 2 If, ] A )4 

where y = minO',,7 2 ) 
A: = max(fc,,fc 2 ) 

^ = P|A y, < 7 2 ) V (J 2 A >, > j 2 )] 

e= [(e, a >k 2 )v (e 2 a&, < & 2 )] 



41 



3)(=,i) A{i} 
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42 a(<, a JJW,WM,U A ) A UU [*i»°°]t>t H 0 . 00 !^) 

43 ®(>vJJW>WMlUfl) A a ^y TF) < a2 [ T [- 0O > O0 ]T > T[- 0O ,7lLa 1 A-,/ J2 A* 1 ]A> 

44 SKm^tJi,,./,] [MJJ*) A U t C a [ Si [^pA]^t[-^«>]t] a > 

45 3)(mi, ai [Jw'Jv*. R'AUa) ^ U Ut^-W^AL*) 

a 2 €{T,r} 

46 3(o,jjw;] 5i ,jM]J A ) a u < 

a 2 ,/f 2 e{T,F} 

47 3( oi [ [z, , y, ] [fc, , /, ] ] A ) A U L [ T [-co, j\ ]^ a ^ , ai aA Ari [i, , /, ] ft > 

a 2 ,/* 2 €{T,F) 

48 ^JJw.WV.Ua) a U < a, U [ Z l ' ^1 ]^ aA [^1 > °°]t ]fi 2 ) 

49 s(si, [jw.L,,*, [*,,/, if, ] A ) a y ujw^w^y 

so ^(MJw.W^iUa) a U C ! [ T [-«,i 1 L 1 ^ V6| [^/,] fl V 

a 2 e{T,F} 

51 3(fi,jjw.WMi]f,]A) a y f u 0 ,. Wi [/„4»,[^',] (i v 

-iaiAa 2 A^ '-i/^A/^AC! 

53 3( di , a , [ r , ft , J\ \ , 6 , [*, , /, ] (> ] A ) A U < 

a 2 ta, A-,a 2 ajt, Ul > °°] T >T [ 00 > A ] a-,& a£, 1ft ) 

a 2 '/>2 e t T > F f 

54 

55 and, l(i,r,j)A |J U U U Span(i\|"). 

i'eS>(i--\j) i'si'ni feSfr,!) j'ej'Dj 
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